{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise 10 - Custom Resources\n",
    "\n",
    "**GOAL:** The goal of this exercise is to show how to use custom resources\n",
    "\n",
    "See the documentation on using Ray with custom resources http://ray.readthedocs.io/en/latest/resources.html#custom-resources.\n",
    "\n",
    "### Concepts for this Exercise - Using Custom Resources\n",
    "\n",
    "We've discussed how to specify a task's CPU and GPU requirements, but there are many other kinds of resources. For example, a task may require a dataset, which only lives on a few machines, or it may need to be scheduled on a machine with extra memory. These kinds of requirements can be expressed through the use of custom resources.\n",
    "\n",
    "Custom resources are most useful in the multi-machine setting. However, this exercise illustrates their usage in the single-machine setting.\n",
    "\n",
    "Ray can be started with a dictionary of custom resources (mapping resource name to resource quantity) as follows.\n",
    "\n",
    "```python\n",
    "ray.init(resources={'CustomResource1': 1, 'CustomResource2': 4})\n",
    "```\n",
    "\n",
    "The resource requirements of a remote function or actor can be specified in a similar way.\n",
    "\n",
    "```python\n",
    "@ray.remote(resources={'CustomResource2': 1})\n",
    "def f():\n",
    "    return 1\n",
    "```\n",
    "\n",
    "Even if there are many CPUs on the machine, only 4 copies of `f` can be executed concurrently.\n",
    "\n",
    "Custom resources give applications a great deal of flexibility. For example, if you wish to control precisely which machine a task gets scheduled on, you can simply start each machine with a different custom resource (e.g., start machine `n` with resource `Custom_n` and then tasks that should be scheduled on machine `n` can require resource `Custom_n`. However, this usage has drawbacks because it makes the code less portable and less resilient to machine failures."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import absolute_import\n",
    "from __future__ import division\n",
    "from __future__ import print_function\n",
    "\n",
    "import ray\n",
    "import time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this exercise, we will start Ray using custom resources."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ray.init(num_cpus=8, resources={'Custom1': 4}, include_webui=False, ignore_reinit_error=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**EXERCISE:** Modify the resource requirements of the remote functions below so that the following hold.\n",
    "- The number of concurrently executing tasks is at most 8 (note that there are 8 CPUs).\n",
    "- No more than 4 copies of `g` can execute concurrently (note that there are only 4 \"Custom1\" resources).\n",
    "- If 4 `g` tasks are executing, then an additional 4 `f` tasks can execute.\n",
    "\n",
    "You should only need to use the `Custom1` resource."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@ray.remote\n",
    "def f():\n",
    "    time.sleep(0.1)\n",
    "\n",
    "@ray.remote\n",
    "def g():\n",
    "    time.sleep(0.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you did the above exercise correctly, the next cell should execute without raising an exception."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "start = time.time()\n",
    "ray.get([f.remote() for _ in range(8)])\n",
    "duration = time.time() - start \n",
    "assert duration >= 0.1 and duration < 0.19, '8 f tasks should be able to execute concurrently.'\n",
    "\n",
    "start = time.time()\n",
    "ray.get([f.remote() for _ in range(9)])\n",
    "duration = time.time() - start \n",
    "assert duration >= 0.2 and duration < 0.29, 'f tasks should not be able to execute concurrently.'\n",
    "\n",
    "start = time.time()\n",
    "ray.get([g.remote() for _ in range(4)])\n",
    "duration = time.time() - start \n",
    "assert duration >= 0.1 and duration < 0.19, '4 g tasks should be able to execute concurrently.'\n",
    "\n",
    "start = time.time()\n",
    "ray.get([g.remote() for _ in range(5)])\n",
    "duration = time.time() - start \n",
    "assert duration >= 0.2 and duration < 0.29, '5 g tasks should not be able to execute concurrently.'\n",
    "\n",
    "start = time.time()\n",
    "ray.get([f.remote() for _ in range(4)] + [g.remote() for _ in range(4)])\n",
    "duration = time.time() - start \n",
    "assert duration >= 0.1 and duration < 0.19, '4 f and 4 g tasks should be able to execute concurrently.'\n",
    "\n",
    "start = time.time()\n",
    "ray.get([f.remote() for _ in range(5)] + [g.remote() for _ in range(4)])\n",
    "duration = time.time() - start \n",
    "assert duration >= 0.2 and duration < 0.29, '5 f and 4 g tasks should not be able to execute concurrently.'\n",
    "\n",
    "print('Success!')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
